Phase diagrams for an evolutionary Prisoner's Dilemma game 
on two-dimensional lattices 
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The effects of payoffs and noise on the maintenance of cooperative behavior are studied in an evolutionary 
Prisoner's Dilemma game with players located on the sites of different two-dimensional lattices. This system 
exhibits a phase transition from a mixed state of cooperators and defectors to a homogeneous one where only 
the defectors remain alive. Using systematic Monte Carlo simulations and different levels of the generalized 
mean-field approximations we have determined the phase boundaries (critical points) separating the two phases 
on the plane of the temperature (noise) and temptation to choose defection. In the zero temperature limit this 
analysis suggests that the cooperation can be sustained only for those connectivity structures where three-site 
clique percolation occurs. 
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In the original (two-player and one-shot) Prisoner's 
Dilemma (PD) game fT,"?] the players should simultaneously 
choose between two options, called defection and coopera- 
tion. The selfish players wish to maximize their own income 
in the knowledge of payoffs dependent on their choices. The 
curiosity of PD game is hidden in the fact that the choice of 
defection yields higher income independently of the partner's 
choice. However, if both players choose defection then their 
individual income is lower than those obtained for mutual co- 
operation when the maximum total payoff is shared equally. 
The rational (intelligent) players cannot resolve this dilemma 
and both of them choose defection (this is the so-called Nash- 
equilibrium in the PD game). At the same time we find many 
examples in the nature where the mutual cooperation (altru- 
ism, ethical norms, etc.) emerges spontaneously among the 
selfish individuals fs*]. In the last decades several mechanisms 
(e.g., kin selection |4], application of retaliating strategies Q], 
and voluntary participation | 6]) are reported which enforce the 
appearance of cooperation in the societies. 

The spatial versions fT^lsl] of the evolutionary PD games can 
explain the maintenance of cooperation for the iterated games 
with a limited range of interaction if the players follow one of 
the two simplest strategies. For the two simplest strategies, de- 
noted shortly as D and C, the player choose always defection 
and cooperation, respectively. In the evolutionary games the 
players wish to maximize their total payoff, coming from PD 
games with the neighbors, by adopting one of the more suc- 
cessful strategies available in their neighborhood. This type 
of dynamics describes the behavior of the geological systems 
controlled by the Darwinian selection jgirioll. 

Following the pioneering work of Nowak et al. ff, "s*] the 
two-strategy spatial evolutionary PD games have already been 
studied by several authors using different evolutionary rules 
on a large class of backgrounds including social networks 

SI12. 13 . ,.14] (for a survey of lattice models see the papers 
[lq.[l7ll and further references therein). In the present pa- 
per our attention is focused on the effect of noise built into 
the dynamical rule. It is turned out that the effect of noise on 
the stationary concentration of cooperators depends strongly 
on the topological features of the neighborhood and the mea- 



sure of cooperation can be enhanced by increasing the noise 
in some cases. 

For this purpose we consider an evolutionary PD game with 
players located on the sites a; of a two-dimensional lattice. 
The players follow one of the above mentioned two strategies 
whose distribution is described by a two-state Potts model, 
i.e., Sx — C or D, where for later convenience the states are 
denoted by the two-dimensional unit vectors. 
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In this notation the total income of player x can be expressed 
as 



Sx+S, 



(2) 



where s+ denotes the transpose of the state vector s^;, the sum- 
mation runs over those four neighbors who the player x plays 
PD game with. Following Nowak et al. L2J the rescaled payoff 
matrix is given as 
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The evolutionary process is governed by random sequential 
strategy adoptions, that is, the randomly chosen player x 
adopts one of the (randomly chosen) neighboring strategy (at 
site y) with a probability depending on the payoff difference 



Wis, 



1 



l + c^p[{Ux-Uy)/K] 



(4) 



where K is the measure of stochastic uncertainties (noise) al- 
lowing the irrational choices ifislfl^ . 

Our analysis will be restricted to two-dimensional lattices 
where the topologically equivalent sites have four neighbors 
(z — 4) as indicated by the edges in Fig. \l\ By this way 
we can avoid the undesired effects due to the variation of the 
number of co-players 1 7, 8, 20, 21]. 

The investigated connectivity structures are the square (1) 
and Kagome (3) lattices, and a square lattice of four-site 
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FIG. 1 : Three two-dimensional lattice stiuctures on which an evolu- 
tionary Prisoner's Dilemma game is studied. 



c ^ —0 which is suggested by Nowak et al. in their pioneer- 
ing work fl\ . 

Figure |2] shows the concentration of cooperators on the 
square lattice when increasing h for three different values of 
K. These data are obtained by Monte Carlo (MC) simulations 
performed on a block of L x L sites under periodic bound- 
ary conditions. The linear size is varied from L — 400 to 
L — 2000. The larger sizes are used in the close vicinity 
of the extinction of cooperators because this critical transition 
belongs to the so called directed percolation (DP) universality 
class L22,2i2i.25J- 



cliques (2). The latter structure consists of four-site cliques 
(within a clique the nodes are linked to each other) whose sites 
are connected only to one external site belonging to the near- 
est clique. These structures can be distinguished topologically 
by considering the connectivity between the triangles (three- 
site cliques). In fact, the square lattice is free of triangles, that 
is, its clustering coefficient C — Q. On the contrary, C = 1/2 
and C = 1/3 for the structures 2 and 3. On the Kagome lat- 
tice percolation of overlapping triangles takes place whereas 
the overlapping triangles form isolated four-site cliques on the 
structure 2. 

In order to investigate the relevance of the mentioned topo- 
logical features first we show the prediction of the classical 
mean-field theory where the state is characterized by the con- 
centration p of cooperators. In this case the average payoff for 
the C and D strategies are 



Uc 



z[p + (1 — p)c] and Ud ~ zpb 



(5) 



The present dynamical rule, represented by the adoption prob- 
ability (|2j, yields the following equation of motion for the 
concentration of cooperators: 



dp 
dt 



= p{l - p)[W{D ^ C) - W{C ^ D)] 



-p{l — p) tanh 



Up-Uc 
2K 



(6) 



According to this differential equation p tends to zero for 
arbitrary value of K as Ud > Uc- Shortly, the cooperators 
become extinct in those systems satisfying the conditions of 
mean-field approximation, e.g., if the temporal co-players are 
chosen randomly or in a system where all the possible pairs 
play a game with each other (infinite range of interaction). 

Here it is worth mentioning that the cooperators also die 
out in the one-dimensional system |22| because for a con- 
fronting cooperator-defector pair the maximum cooperator's 
payoff (1 + c) is always less than the minimum defector's pay- 
off (5). 

For higher dimension, however, the cooperator can receive 
support from more than one neighboring cooperators and its 
total income can exceed the neighboring defector's income. 
For such a connectivity structure the cooperation can be sus- 
tained within a region of b (and c) dependent on the value of 
noise (K). This work is addressed to quantify the regions of 
the b-K parameter plain where cooperation can emerge. For 
sake of simplicity, our analysis will be restricted to the limit 
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FIG. 2: Monte Carlo results for the concentration of cooperators vs. 
b for three different temperatures: K = 0.1 (pluses), 0.4 (squares), 
and 1.2 (diamonds) on the square lattice. 

In the stationary state the concentration of cooperators is 
independent of the initial state and decreases monotonously if 
b is increased. Above a threshold value (6 > bcr), however, 
the C strategies always die out and the system remains in the 
homogeneous D state for ever. The value of bcr is determined 
for many different values of K and the results of the system- 
atic MC simulations are summarized in Fig.|3] Notice that bcr 
reaches its maximum value at about K — 0.32 and bcr{K) 
tends to 1 if K goes to either or oo. Henceforth this plot 
is considered as a phase diagram because the cooperators can 
survive only below the bcr{K) curve indicated by the solid 
line connecting the MC data in Fig.|3l 

This phase diagram differs significantly from those pre- 
dicted by the above mentioned mean-field approximation 
[b')!^^^\K) = 1]. More adequate theoretical results are ex- 
pected when using the pair approximation detailed in fT%. 
This approach is able to describe the coexistence of the C and 
D strategies, however, the value of bcr is significantly overes- 
timated as shown by the dashed line (in Fig.|3l which goes to 
2 in the limit K ^ {). This serious shortage can be reduced 
by using the more sophisticated extensions of this technique 
when all possible configuration probabilities are determined 
on larger clusters. The generalization is straightforward from 
two-site clusters (pair approximation) to larger blocks (the 
essence of this method is briefly described in L26i. i27.1 ). Ne- 
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FIG. 3: Critical value of 6 as a function of temperature on the square 
lattice. Symbols come from Monte Carlo simulations, the dashed, 
dotted, and dashed-dotted lines represent the prediction of general- 
ized mean-field approximation for 2-, 2 x 2-, and 3 x 3-site clusters. 

glecting the technical details now we report only the results of 
this calculation for the levels of 2 x 2- and 3 x 3-site clusters. 
In both cases the calculations reproduce the main qualitative 
features (see Fig.|3}, that is, bcr{K) has a maximum at a finite 
K and her = 1 in the limits K ^ and oo. Evidently, the 
accuracy of this approach is improved when choosing larger 
and larger clusters and the strikingly large deviations between 
the prediction of different levels refer to the importance of the 
complex short range order affected by the local topological 
features. 




FIG. A: h — K phase diagram on the structure 2 illustrated in Fig.Q 
Symbols denote the MC data. The dotted and dashed lines illustrate 
the phase boundary between the D and (C + D) phases as predicted 
by generalized mean-field approximations for the four- and eight-site 
clusters shown at the top. 

This behavior indicates the existence of a noise level pro- 



viding the highest measure of cooperation at a fixed b for a 
square lattice connectivity structure. This means furthermore 
that one can observe two subsequent phase transitions (both 
are belonging to the DP universality class) if K is increased 
from zero for a fixed value of temptation h < max(6cr). 

On structure 2 the results of MC simulations are very sim- 
ilar to those found on the square lattice (the differences are 
comparable to the symbol size) as shown in Fig. |4] In con- 
trary to the square lattice, the four-site approximation overes- 
timates the results of MC simulations obtained on structure 2. 
At the same time the prediction of the eight-site approxima- 
tion fits very well to the MC data for low noises (K < 0.3). 
It is suspected that the prediction of eight-site approximation 
(particularly for large K vales) can be observed on such non- 
spatial structures where four-site cliques are substituted for 
the nodes of a random regular graph (or Bethe lattice) with a 
degree of four. (Notice, that the eight-site cluster is equivalent 
to a pair of four-site cliques and the pair approximation seems 
to be more correct for the Bethe lattice due to the absence of 
loops (3). 

In contrary to the above phase diagrams, a qualitatively dif- 
ferent behavior is observed on the Kagome lattice as illus- 
trated in Fig.|5] The most striking feature is that here the crit- 
ical value of b decreases monotonously if K is increased and 
bcr{K — 0) = 3/2 in agreement with the prediction of the 
three- and five-site approximations. 




FIG. 5: Phase diagram on the Kagome lattice [structure 3 in Fig.fTl. 
Symbols denote the MC data. The dotted and dashed lines illustrate 
the phase boundary suggested by the three- and five-site approxima- 
tion on the clusters shown at the top. 

In order to deduce a general picture about the relevant topo- 
logical features supporting the maintenance of cooperation in 
the low noise limit (for the suggested dynamics) we have be- 
gun to study several other connectivity structures. Accord- 
ing to the preliminary results the latest phase diagram (see 
Fig. E} is reproduced qualitatively on the square lattice with 
first and second neighbor interactions {z = 8), on the trian- 
gular lattice (z — 6), and on the body centered cubic lattice 
(z = 8). In agreement with our expectation, the prediction 
of the five-site approximation (shown in Fig.|5j is reproduced 
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very well by the MC results obtained on the random regular 
structure (z — 4) constructed from one-site overlapping trian- 
gles. For all these structures the overlapping triangles (three- 
site cliques) span the whole system. It would be interesting to 
check the emergence of cooperation (in the K limit) on 
other networks where clique percolation takes place 112^13(11 . 
We have to emphasize, however, that the cooperation is not 
favored within the large cliques according to the mean-field 
arguments mentioned above. This might be another reason 
why the cooperation vanishes on the structure 2 in the K 
limit. In agreement with the above conjecture the cooperators 
die out for vanishing K on the cubic (z — 6) and honeycomb 
(z = 3) lattices. Besides it, the one-dimensional lattice with 
first and second neighbor interactions {z = 4) represents an 
exception (because it inherits the one-dimensional features on 
large scales) exhibiting a sharp transition between the homo- 
geneous states (from C to if & is increased for a fixed K). 

In summary, we have studied systematically the effect of 
noise K (allowing irrational strategy adoptions) and temp- 
tation b to choose defection on the measure of cooperation 
in an evolutionary Prisoner's Dilemma game for such two- 
dimensional lattice structures where the number of neighbors 
is fixed, z — i. For the investigated dynamical rule two ba- 
sically different behaviors can be distinguished when varying 
the connectivity structures. In the first case the cooperators 
die out in the zero noise limit and the maintenance of coop- 



eration can be optimized by choosing a suitable level of noise 
for any fixed value of temptation if b < max(6cr). In the 
second case the highest measure of cooperation occurs for the 
lowest temptation {b = 1) and noise K — and the critical 
value of b decreases if K is increased. It is conjectured that 
the second behavior occurs for all the d-dimensional, d > 2, 
or non-spatial (e.g., Bethe lattice or random regular graphs) 
connectivity structures where the overlapping triangles span 
the whole system. This indicates that the percolation of the 
overlapping triangle in the connectivity structure can provide 
the optimum topological condition for the maintenance of co- 
operation in the situations of multi-agent Prisoner's Dilemma. 

In the last years several al gori thms were introduced to cre- 
ate a large class of networks ll3lll32l t33] and very recently the 
games are also suggested to control the evolution of a network 
L34. .351. The above results raise the chance that similar evo- 
lutionary PD games (in the zero noise limit) can be utilized to 
control the creation of networks of percolating triangles. 
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